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METHOD AND SYSTEM FOR PANORAMIC IMAGE GENERATION USING 
CLIENT-SERVER ARCHITECTURE 

CROSS-REFERENCE TO RELATED APPLICATIONS 
5 This application claims the benefit of U.S. Provisional Application Serial No. 

60/271,155 filed February 24, 2001; U.S. Provisional Application Serial No. 60/271,154 filed 
February 24, 2001; U.S. Provisional Application Serial No. 60/315,744 filed August 29, 
2001; and U.S. Provisional Application Serial No. 60/346,717 filed January 7, 2002. 

10 FIELD OF THE INVENTION 

The present invention relates to photographic imaging, and more particularly 
to method and system for panoramic image generation using client-server architecture. 

BACKGROUND INFORMATION 
Recent work has shown the benefits of panoramic imaging, which is able to 
15 capture a large azimuth view with a significant elevation angle. If instead of providing a 

small conic section of a view, a camera could capture an entire half-sphere at once, several 
advantages could be realized. Specifically, if the entire environment is visible at the same 
time, it is not necessary to move the camera to fixate on an object of interest or to perform 
exploratory camera movements. This also means that the view can be supplied to multiple 
20 viewers who can independently choose their own view of the environment. A viewer can 

choose to look to the left and the other can choose to look to the right. 

A panospheric camera is a device that captures light from all directions (i.e., 
360 degrees), either as stills or as a continuous video stream. The raw images from such a 
device can be geometrically transformed to synthesize a conventional camera view in any 
25 direction. One method for constructing a panospheric camera is to attach a curved mirror to 

an imaging device, such as a still camera or video camera. The mirror gathers light from all 
directions and re-directs it to the camera. Both spherical and parabolic mirrors have been 
used in panoramic imaging systems. 

Regardless of the type of device or camera that is used to capture the raw 
30 images, the raw images must be converted or processed into panoramic images. The 

conversion or processing may include, for example, utilizing unwarping software for 
converting the raw images into a panoramic image format suitable for viewing, such as the 
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panoramic Quicktime VR format. Once the images are converted to the proper panoramic 
format, a user can view them interactively on a computer. 

Currently, to convert the images captured by a camera, the images are first 
imported to a user computer. The images are then processed to generate the panoramic 
5 images on the user computer using processing or unwarping software that is installed on the 

user's computer. The panoramic images may then be displayed or used on the user computer. 
However, this process requires that the user purchase the processing software, load the 
processing software on the user computer, and then process the images. This process, 
therefore, can be an expensive and difficult process for the user to employ. 
10 There is identified, therefore, a need for an improved method and system for 

panoramic image generation that overcomes disadvantages, shortcomings, or limitations of 
known methods and systems for panoramic image generation. 

SUMMARY OF THE INVENTION 
15 Embodiments of the invention meet the identified need, as well as other needs, 

as will be more fully understood following a review of the specification and drawings. 

In accordance with an aspect of the invention, in a client-server computer 
network, a computer-assisted method for generating a panoramic image is provided. The 
method includes transmitting a raw photographic image to the server computer and 
20 processing the raw photographic image on the server computer to obtain the panoramic 

image. 

In accordance with another aspect of the invention, in a client-server computer 
network, an apparatus for generating a panoramic image is provided. The apparatus includes 
means for transmitting a raw photographic image to the server computer and means for 
25 processing the raw photographic image on the server computer to obtain the panoramic 

image. 

In accordance with yet another aspect of the invention, in a client-server 
computer network, a computer readable medium having stored thereon instructions which, 
when executed by a processor, cause said processor to perform certain steps is provided. The 
30 steps include transmitting a raw photographic image to the server computer and processing 

the raw photographic image on the server computer to obtain the panoramic image. 
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In accordance with yet an additional aspect of the invention, in a client-server 
computer network, a system for generating a panoramic image includes a processor and a 
memory device having data and instructions stored thereon which, when executed by the 
processor, cause the processor to perform certain steps. The steps include transmitting a raw 
5 photographic image to the server computer and processing the raw photographic image on 

the server computer to obtain the panoramic image. 

These and other aspects of the present invention will be more apparent from 
the following description. 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates a raw image in spherical or annular form which may be 

processed in accordance with the invention to obtain a panoramic image thereof. 

Figure 2 illustrates a 360 degrees cylindrical panoramic view of the raw image 

shown in Figure 2. 

15 Figure 3 illustrates an embodiment of the invention for panoramic image 

generation. 

Figure 4 illustrates an additional embodiment of the invention for panoramic 
image generation. 

Figure 5 illustrates yet an additional embodiment of the invention for 
20 panoramic image generation. 

Figure 6 illustrates a further embodiment of the invention for panoramic image 

generation. 

DETAILED DESCRIPTION OF THE INVENTION 
25 Embodiments of the invention provide a method and system for panoramic 

image generation using client-server architecture. In addition, embodiments of the invention 
provide an apparatus and a computer readable medium for panoramic image generation using 
client-server architecture. 

As used herein, the term "panoramic image" or "panoramic view" generally 
30 refers to wide angle photographic images taken from a field of view of up to 360° around a 

principal axis, wherein the images also may cover a vertical field of view typically ranging 
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from 0.1° to 180°. These panoramic images can be stored in several different formats such as, 
for example, Apple Quicktime VR or Microsoft Surround Video. 

As used herein, the term "raw image" or "captured image" generally refers to a 
spherical image, annular image, or other type image that is taken by a camera or similar 
5 device that must be converted, processed, or unwarped in order to obtain a panoramic image 

corresponding thereto. 

As used herein, the term "camera" generally refers to any type device capable 
of capturing a photographic image, either still or video, as a raw image for processing to 
obtain a panoramic image. 

10 As used herein, the term "client" can be any computer that a user employs to 

receive, transmit, capture, and/or view the panoramic image. 

As used herein, the term "server" can be any computer where the adjustment 
of the raw image and unwarping of the raw image are done. The server may be connected to 
the client using either indirect or direct connection which can be temporary or permanent. 

15 It is to be understood that the figures and descriptions of the present invention, 

and specifically embodiments thereof, have been simplified to illustrate elements that are 
relevant for a clear understanding of the present invention while eliminating, for purposes of 
clarity, other elements found in a typical computer network. For example, specific operating 
system details and modules are not shown. Also, specific network items such as network 

20 routers are not shown. In addition, the computers used in conjunction with the present 

invention may be any type of suitable computer such as, for example, an IBM compatible 
personal computer, a mainframe computer, an Apple Macintosh computer, a computer 
workstation, a handheld computing device or other types of computing and electronic 
devices. Those of ordinary skill in the art will recognize that other elements are desirable 

25 and/or required to produce an operational system incorporating the present invention. 

However, because such elements are well known in the art, and because they do not facilitate 
a better understanding of the present invention, a detailed discussion of such elements is not 
provided herein. 

Figure 1 illustrates a raw image that is captured for processing in accordance 
30 with the invention. The raw image is obtained from a camera that is capable of capturing 

such images for conversion to a panoramic image. The camera may include, for example, a 
panoramic mirror system used in conjunction therewith such as, for example, the arrangement 
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described in United States Patent Application Serial No. filed on 

and entitled Improving Usable Resolution Of Panoramic Mirrors, which is 

commonly owned by the assignee of the present application and the entire disclosure of 
which is hereby incorporated by reference. It will be appreciated that other panoramic mirror 
5 arrangements and similar type systems for obtaining a raw image for conversion into a 

panoramic image may be used in conjunction with the present invention. 

Figure 2 illustrates a panoramic image in a 360 degrees cylindrical image 
corresponding to the raw image set forth in Figure 1. The panoramic image set forth in 
Figure 2 is the resulting image following the processing of the raw image. A more common 

10 way to display the resulting panoramic image is by using applications such as the Apple 

Quicktime. This is done by converting the panoramic image to the appropriate format, which 
in the case of Quicktime VR, the Quicktime VR format. Using such an application, the 
viewer can interactively view the panoramic image. The viewer can choose the viewing 
direction and the field of view. Quicktime VR or another player then generates a perspective 

15 image of the area that the viewer wants to look at. 

Figure 3 illustrates a first embodiment of the invention for generating a 
panoramic image. Specifically, a camera 10 is used for capturing a raw image. The raw 
image is then imported or transmitted, as illustrated at 1 1, from the camera to a user or client 
computer 12. The raw image may be downloaded from the camera 10 to the client computer 

20 12 by a physical connection between the camera 10 and the client computer 12, by storing the 

captured image on a recording medium and then the client computer 1 2 reading the data from 
the recording medium, or by a wireless transmission from the camera 10 to the client 
computer 12. 

Once the raw photographic image is resident on the client computer 12, the 
25 image is transmitted, as illustrated at 14, to a server computer 16. The images may be 

transmitted from the client computer 12 to the server computer 16 using, for example, an 
Internet connection therebetween, a wireless connection, a phone line, or other suitable 
networking medium. Furthermore, the images may be transmitted using various network 
protocols, including e-mail, File Transfer Protocol (FTP), Hypertext Transfer Protocol 
30 (HTTP), or other suitable networking protocols. 

Once the raw images have been transmitted to the server computer 1 6 and are 
resident thereon, the server computer 16 provides a means for processing the raw 
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photographic image to obtain the panoramic image. One aspect of processing the images on 
the server computer 16 may include utilizing unwarping software installed on the server 
computer 1 6 to unwarp the raw images. The unwarping of the image includes converting or 
processing the raw image to obtain the panoramic image corresponding thereto. The 
5 unwarping software may include any such known software for unwarping a raw image to 

obtain a panoramic image or may include, for example, the unwarping software disclosed in 

United States Patent Application Serial No. filed on and 

entitled Method And Apparatus For Processing Photographic Images, which is commonly 
owned by the assignee of the present application and the entire disclosure of which is hereby 

10 incorporated by reference. 

Another aspect of processing the images on the server computer 16 may 
include adjusting the raw image to orient the raw image either prior to or after obtaining the 
panoramic image. For example, the adjusting of the raw image may include establishing a 
center point and radius of the raw image as related to the camera and panoramic mirror 

15 arrangement used to obtain the raw image, providing characteristics of the panoramic mirror 

arrangement such as elevation or vertical field of view, adjusting for incorrect picture angle 
or correcting the white balance and exposure which may have resulted when the raw image 
was originally captured by the camera 10. 

Still referring to Figure 3, once the raw image has been processed to obtain a 

20 corresponding panoramic image, the panoramic image may then be transmitted, as illustrated 

at 18, back to the client computer 12. The panoramic image may be transmitted from the 
server computer 16 to the client computer 12 in a similar manner as described herein for 
transmitting the raw images from the client computer 12 to the server computer 16. 

Once the panoramic images have been transmitted back to the client computer 

25 12 and are resident thereon, a user may then display, view and/or use the processed 

panoramic images as desired. The client computer 12 should have installed thereon, software 
capable of viewing the panoramic images, such as Quicktime VR software available from 
Apple Computer, Inc. 

Figure 4 illustrates an additional embodiment of the invention. This 

30 embodiment is similar to that shown in Figure 3 and described herein, wherein a camera 110 

is used to capture a raw image and the image is then imported or transmitted, as illustrated at 
1 1 1, to a client computer 112. The raw images are then transmitted from the client computer 
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112, as illustrated at 114, to a server computer 116. The server computer 116 includes 
suitable software installed thereon for processing the raw image to, for example, obtain a 
panoramic image, as described herein. The panoramic image is then transmitted, as 
illustrated at 118, to an additional viewing computer or web server 120, rather than being 
5 transmitted back to the client computer 112. The panoramic image may then be viewed 

and/or used on the viewing computer/web server 120. In addition, the panoramic image can 
be stored on the viewing computer/web server 120 in order that the panoramic image may be 
accessed through a network, e.g. the panoramic image may be embedded in a web page or 
made available for viewing on a particular web page. 

10 Figure 5 illustrates yet an additional embodiment of the invention for 

generating a panoramic image. Specifically, in this embodiment, a camera 212 is used to 
capture a raw image. Rather than transmitting the captured image from the camera to a user 
or client computer as illustrated in Figures 3 and 4, the raw image may be transmitted, as 
illustrated at 214, directly to a server computer 216. This transmission may be performed by 

15 utilizing a camera, such as a digital camera, with the capability to transmit the images over a 

network using, for example, a wireless connection or a landline network. 

The server computer 216 then processes the raw image to obtain a panoramic 
image, in a manner similar to the processing described herein in relation to the embodiments 
set forth in Figures 3 and 4. The server computer 216 then transmits the panoramic image, as 

20 illustrated at 218, to a viewing computer/web server 220. The viewing computer/web server 

220 may then be utilized for further use of the panoramic image, as described herein in 
relation to the embodiment set forth in Figure 4. 

Figure 6 illustrates yet an additional embodiment of the invention. 
Specifically, a camera 312 is used to capture a raw image and then to transmit the raw image, 

25 as illustrated at 314, directly to a viewing/server computer 316. Similar to the embodiment 

set forth in Figure 5, the camera 312 is preferably a digital camera with the capability to 
transmit the raw images over a network to the viewing/server computer 316. The 
viewing/server computer 316 is then capable of processing the raw image to obtain a 
panoramic image, as described herein for other embodiments. The viewing/server computer 

30 316 is also configured to allow the panoramic image to be viewed directly thereon or to place 

the processed panoramic image on a network for viewing by a remote computer. In addition, 
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the viewing/server computer 316 is configured to have the panoramic image embedded in a 
web page for viewing on a computer network. 

Accordingly, it will be appreciated that the invention as illustrated, for 
example, by the embodiments set forth herein provides for panoramic image generation using 
5 various client-server architectures. To avoid the complexities and expense of requiring a user 

to install on a user or client computer processing software for converting or processing a raw 
image to a panoramic image, the present invention provides for transmitting the raw images 
to a remote computer, such as the server computer 16, 116, 216, or the viewing/server 
computer 316, where the processing of the raw image to obtain a panoramic image is 

10 performed. The resulting panoramic image is then transmitted back to a user or client 

computer, such as client computer 12 or 1 12 or to an additional computer such as viewing 
computer/web server 220 for displaying or using the panoramic image as desired. In 
addition, the panoramic image may be placed on a computer network or a web server to allow 
the panoramic image to be viewed over the network by multiple viewers. An advantage of 

15 the invention, therefore, is that no processing software is necessary on the user's computer. 

Rather, the user can transmit the raw image to a server computer and have the raw image 
automatically processed to obtain a panoramic image corresponding thereto. The 
processing/unwarping on the server can be an automatic, unattended process by using 
software that can automatically make adjustments to the raw image, and then unwarp the raw 

20 image. The transmission of the raw image and the panoramic image from one computer to 

another may be performed using standard available networking software and protocols as 
described herein. 

Another advantage of the invention is that the user can include along with the 
raw image being transmitted additional information regarding the image. For example, the 

25 user may send an e-mail to the server computer which includes the web address to which the 

processed panoramic image should be forwarded, and the e-mail may include the raw images 
as attachments thereto. The server computer receives the raw images, automatically 
processes them to produce panoramic images, and then places the panoramic image at the 
desired web address. A practical example of such a process is a real estate agent taking a 

30 photograph to obtain a raw image and then e-mailing the raw image as attachments to the 

server computer. In the title or body of the images, the real estate agent can include the web 
address for the particular property or address of the property that corresponds to the 
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photographic image. The server computer receives the images, processes the images to 
obtain a panoramic image, and then places the image on the correct web page based on the 
information that the real estate agent included in the e-mail. 

Another advantage of the invention is essentially instantaneous generation of 

5 panoramic images by employing the embodiments where a camera having wireless 

connection capabilities is used to transmit the raw image directly to a server computer for 
processing to obtain a panoramic image. Thus, the user can use the camera to obtain the raw 
image and send them directly to the server without having to use a computer at all. The 
Ricoh RDC-i700 is an example of a digital camera with the ability to send its pictures directly 

10 to a server. 

Whereas particular embodiments of this invention have been described above 
for purposes of illustration, it will be evident to those skilled in the art that numerous 
variations of the details of the present invention may be made without departing from the 
invention as defined in the appended claims. 

15 
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